Remote Sensing Image Retrieval Research Articles

A Transformer and Visual Foundation Model-Based Method for Cross-View Remote Sensing Image Retrieval

Abstract. Retrieving UAV images that lack POS information with georeferenced satellite orthoimagery is challenging due to the differences in angles of views. Most existing methods rely on deep neural networks with a large number of parameters, leading to substantial time and financial investments in network training. Consequently, these methods may not be well-suited for downstream tasks that have high timeliness requirements. In this work, we propose a cross-view remote sensing image retrieval method based on transformer and visual foundation model. We investigated the potential of visual foundation model for extracting common features from cross-view images. Training is only conducted on a small, self-designed retrieval head, alleviating the burden of network training. Specifically, we designed a CVV module to optimize the features extracted from the visual foundation model, making these features more adept for cross-view image retrieval tasks. And we designed an MLP head to achieve similarity discrimination. The method is verified on a publicly available dataset containing multiple scenes. Our method shows excellent results in terms of both efficiency and accuracy on 15 sub-datasets (10 or 50 scene categories) derived from the public dataset, which holds practical value in engineering applications with streamlined scene categories and constrained computational resources. Furthermore, we initiated a comprehensive discussion and conducted ablation experiments on the network design to validate its efficacy. Additionally, we analyzed the presence of overfitting within the network and deliberated on the limitations of our study, proposing potential avenues for future enhancements.

Zero-Shot Sketch-Based Remote-Sensing Image Retrieval Based on Multi-Level and Attention-Guided Tokenization

Effectively and efficiently retrieving images from remote-sensing databases is a critical challenge in the realm of remote-sensing big data. Utilizing hand-drawn sketches as retrieval inputs offers intuitive and user-friendly advantages, yet the potential of multi-level feature integration from sketches remains underexplored, leading to suboptimal retrieval performance. To address this gap, our study introduces a novel zero-shot, sketch-based retrieval method for remote-sensing images, leveraging multi-level feature extraction, self-attention-guided tokenization and filtering, and cross-modality attention update. This approach employs only vision information and does not require semantic knowledge concerning the sketch and image. It starts by employing multi-level self-attention guided feature extraction to tokenize the query sketches, as well as self-attention feature extraction to tokenize the candidate images. It then employs cross-attention mechanisms to establish token correspondence between these two modalities, facilitating the computation of sketch-to-image similarity. Our method significantly outperforms existing sketch-based remote-sensing image retrieval techniques, as evidenced by tests on multiple datasets. Notably, it also exhibits robust zero-shot learning capabilities in handling unseen categories and strong domain adaptation capabilities in handling unseen novel remote-sensing data. The method’s scalability can be further enhanced by the pre-calculation of retrieval tokens for all candidate images in a database. This research underscores the significant potential of multi-level, attention-guided tokenization in cross-modal remote-sensing image retrieval. For broader accessibility and research facilitation, we have made the code and dataset used in this study publicly available online.

Blockchain-Based Method for Spatial Retrieval and Verification of Remote Sensing Images.

Remote sensing image is a vital basis for land management decisions. The protection of remote sensing images has seen the application of blockchain's notarization function by many scholars. Yet, research on efficient retrieval of such images on the blockchain remains sparse. Addressing this issue, this paper introduces a blockchain-based spatial index verification method using Hyperledger Fabric. It linearizes the spatial information of remote sensing images via Geohash and integrates it with LSM trees for effective retrieval and verification. The system also incorporates IPFS as an underlying storage unit for Hyperledger Fabric, ensuring the safe storage and transmission of images. The experiments indicate that this method significantly reduces the latency in data retrieval and verification without impacting the write performance of Hyperledger Fabric, enhancing throughput and providing a solid foundation for efficient blockchain-based verification of remote sensing images in land registry systems.

EdgeShield: Attack resistant secure and privacy-aware remote sensing image retrieval system for military and geological applications using edge computing

EdgeShield: Attack resistant secure and privacy-aware remote sensing image retrieval system for military and geological applications using edge computing

An Adaptive Weighted Method for Remote Sensing Image Retrieval with Noisy Labels

Due to issues with sample quality, there is an increasing interest in deep learning models that can handle noisy labels. Currently, the optimal way to deal with noisy labels is by combining robust active and passive loss functions. However, the weighting parameters for these functions are typically determined manually or through a large number of experimental iterations, and even the weighting parameters change as the dataset and the noisy rate change. This can lead to suboptimal results and be time-consuming. Therefore, we propose an adaptively weighted method for the combined active passive loss (APL) in remote sensing image retrieval with noisy labels. First, two metrics are selected to measure the noisy samples: the ratio of the entropy to the standard deviation and the difference of the predicted probabilities. Then, an adaptive weighted learning network with a hidden layer is designed to dynamically learn the weighting parameters. The network takes the above two metrics as inputs and is trained concurrently with the feature extraction network in each batch, without significantly increasing the computational complexity. Extensive experiments demonstrate that our improved APL method outperforms the original manually weighted APL method and other state-of-the-art robust loss methods while saving the time on manual parameter selection.

Score-ranking smooth average precision loss for remote sensing image retrieval

ABSTRACT In remote sensing image retrieval task, deep metric learning loss is an effective method for learning semantic similarity relationships between images. However, most deep metric learning losses treat different ranked images equally. Therefore, we propose a novel score-ranking smooth average precision (SRSAP) loss to improve the retrieval performance. Firstly, the score-ranking (SR) loss builds an ensemble-based similarity structure by using all images. Secondly, we utilize smooth AP to assign more weight to higher-ranked images. Finally, the joint loss function is utilized to fine-tune the network. The experiments show that SRSAP loss is distinctly superior to usual deep metric losses.

Integrating Multisubspace Joint Learning With Multilevel Guidance for Cross-Modal Retrieval of Remote Sensing Images

Integrating Multisubspace Joint Learning With Multilevel Guidance for Cross-Modal Retrieval of Remote Sensing Images

Integration of statistical parameters-based colour-texture descriptors for radar remote sensing image retrieval applications

Integration of statistical parameters-based colour-texture descriptors for radar remote sensing image retrieval applications

Integration of statistical parameters-based colour-texture descriptors for radar remote sensing image retrieval applications

Integration of statistical parameters-based colour-texture descriptors for radar remote sensing image retrieval applications

Remote Sensing Image Retrieval Algorithm for Dense Data

With the rapid development of remote sensing technology, remote sensing products have found increasingly widespread applications across various fields. Nevertheless, as the volume of remote sensing image data continues to grow, traditional data retrieval techniques have encountered several challenges such as substantial query results, data overlap, and variations in data quality. Users need to manually browse and filter a large number of remote sensing datasets, which is a cumbersome and inefficient process. In order to cope with these problems of traditional remote sensing image retrieval methods, this paper proposes a remote sensing image retrieval algorithm for dense data (DD-RSIRA). The algorithm establishes evaluation metrics based on factors like imaging time, cloud coverage, and image coverage. The algorithm utilizes the global grids to create an ensemble coverage relation between images and grids. A locally optimal initial solution is obtained by a greedy algorithm, and then a local search is performed to search for the optimal solution by combining the strategies of weighted gain-loss scheme and novel mechanism. Ultimately, it achieves an optimal coverage of remote sensing images within the region of interest. In this paper, it is shown that the method obtains a smaller number of datasets with lower redundancy and higher data utilization and ensures the data quality to a certain extent in order to accurately meet the requirements of the regional coverage of remote sensing images.

Deep global semantic structure-preserving hashing via corrective triplet loss for remote sensing image retrieval

With the explosive increase of remote sensing data, how to search for remote sensing data quickly and accurately in a vast dataset is an incredibly critical matter for research subjects. The deep hashing method has become the dominant method for remote sensing image retrieval because of its low-cost storage and high-speed retrieval. However, for the reason of the limitation of fixed convolutional kernels, deep hashing frameworks based on Convolutional Neural Networks (CNNs) fail to obtain the global semantic information well, which leads to the generation of suboptimal solutions. Furthermore, existing hashing methods commonly employ the random sampling strategy or hardest sample mining to build training batches, resulting in bad local minima. To remedy these problems, a novel Deep Global Semantic Structure-preserving Hashing framework via corrective triplet loss (DGSSH) is proposed for remote sensing image retrieval to learn a discriminative and stable embedding space, achieving intra-class confusion and inter-class diversity. Specifically speaking, the feature extraction module based on Swim Transformer architecture is developed to capture global semantic information and multiscale features from remote sensing images. Based on a distribution matching constraint, the corrective triplet loss for deep hashing schemes is designed to reduce the distribution shift caused by the random selection or hardest sample mining. Meanwhile, to reduce the time overhead of the model, the asymmetric learning strategy is employed to perform effective compact representation learning. Numerous experiments have been carried out on three publicly available benchmarks, which indicates that the proposed DGSSH framework could achieve optimal performance for remote sensing image retrieval applications. The source code of our DGSSH framework is hosted at https://github.com/QinLab-WFU/DGSSH.git.

Multi-Scale Feature Fusion Based on PVTv2 for Deep Hash Remote Sensing Image Retrieval

For high-resolution remote sensing image retrieval tasks, single-scale features cannot fully express the complexity of the image information. Due to the large volume of remote sensing images, retrieval requires extensive memory and time. Hence, the problem of how to organically fuse multi-scale features and enhance retrieval efficiency is yet to be resolved. We propose an end-to-end deep hash remote sensing image retrieval model (PVTA_MSF) by fusing multi-scale features based on the Pyramid Vision Transformer network (PVTv2). We construct the multi-scale feature fusion module (MSF) by using a global attention mechanism and a multi-head self-attention mechanism to reduce background interference and enhance the representation capability of image features. Deformable convolution is introduced to address the challenge posed by varying target orientations. Moreover, an intra-class similarity (ICS) loss is proposed to enhance the discriminative capability of the hash feature by minimizing the distance among images of the same category. The experimental results show that, compared with other state-of-the-art methods, the proposed hash feature could yield an excellent representation of remote sensing images and improve remote sensing image retrieval accuracy. The proposed hash feature can gain an increase of 4.2% and 1.6% in terms of mAP on the UC Merced and NWPU-RESISC45 datasets, respectively, in comparison with other methods.

Impact of Feature Representation on Remote Sensing Image Retrieval

Remote sensing images are acquired using special platforms, sensors and are classified as aerial, multispectral and hyperspectral images. Multispectral and hyperspectral images are represented using large spectral vectors as compared to normal Red, Green, Blue (RGB) images. Hence, remote sensing image retrieval process from large archives is a challenging task.  Remote sensing image retrieval mainly consist of feature representation as first step and finding out similar images to a query image as second step. Feature representation plays important part in the performance of remote sensing image retrieval process. Research work focuses on impact of feature representation of remote sensing images on the performance of remote sensing image retrieval. This study shows that more discriminative features of remote sensing images are needed to improve performance of remote sensing image retrieval process.

An Intra-Class Ranking Metric for Remote Sensing Image Retrieval

With the rapid development of internet technology in recent years, the available remote sensing image data have also been growing rapidly, which has led to an increased demand for remote sensing image retrieval. Remote sensing images contain rich visual and semantic features, and have high variability and complexity. Therefore, remote sensing image retrieval needs to fully utilize the information in the images to perform feature extraction and matching. Metric learning has been widely used in image retrieval as it can train embedding spaces with high discriminability. However, existing deep metric learning methods learn embedding spaces with high discriminability by maximizing the differences between classes, while ignoring inherent intra-class differences during the learning process. In this paper, we design a new sample generation mechanism to generate samples from positive samples that meet the boundary constraints, thus obtaining quantifiable intra-class differences from real positive samples. Based on the sample generation relationship, we use a self-supervised approach to design an intra-class ranking loss function, which improves the discriminability of the generated embedding space for samples of the same class and maintains their ranking relationship in the embedding space. Moreover, this loss function can be easily combined with existing deep metric learning methods. Our aim is to help the network to better extract features and further improve the performance of remote sensing image retrieval through the sample generation mechanism and intra-class ranking loss. Finally, we conduct extensive experiments on multiple remote-sensing image datasets using multiple evaluation metrics such as mAP@K, which demonstrate that using the sample-generated intra-class ranking loss function can effectively improve the performance of remote sensing image retrieval.

A triplet graph convolutional network with attention and similarity-driven dictionary learning for remote sensing image retrieval

With the explosion in the volume of collected high-resolution aerial image data, the development of effective image retrieval methods for remote sensing (RS) has become a popular area of research. The problem of content-based image retrieval (CBIR) for high-resolution remote sensing (HRRS) requires robust feature extraction and representation to extract high-level semantics followed by a similarity measurement method to compute the similarity between images based on these semantics. In this paper, we propose a novel sparse graph learning-based triplet network for deep metric learning-based HRRS CBIR. First, the graph representations of the aerial images are constructed via local feature descriptors and unsupervised segmentation that provide an efficient graph-structured representation of the images. Then, a novel attention graph convolutional network is developed to extract high-level semantics from the spatial patterns of the images’ segmented regions by aggregating region-node features. The attention mechanism and the pyramid pooling structure of the graph network help the proposed model extract powerful spatial features by preserving the structural information of the images while reducing the data’s dimensionality. Finally, a novel task-driven dictionary learning (TDDL) method based on triplet loss constructs a sparse metric space for the similarity measurement of the images in which the computed sparse codes for the images promote intra-class similarity between same-class images and inter-class dissimilarity between images of different classes. The proposed sparse features reduce the dimensionality of model parameters which mitigates overfitting and improves the generalization of the overall framework for deep metric learning. Our TDDL method adopts a bi-level optimization strategy so that the dictionary can be trained alongside the parameters of the graph representation network in an end-to-end fashion. Finally, we conduct extensive evaluations of the performance of our proposed architecture on real-world datasets compared to state-of-the-art models and obtain superior results.

Content-Based remote sensing image retrieval method using adaptive tetrolet transform based GLCM features

This paper proposed a novel texture feature extraction technique for radar remote sensing image retrieval application using adaptive tetrolet transform and Gray level co-occurrence matrix. Tetrolets have provided fine texture information in the radar image. Tetrominoes have been employed on each decomposed radar image and best pattern of tetrominoes has been chosen which represents the better radar image geometry at each decomposition level. All three high pass components of the decomposed radar image at each level and low pass component at the last level are considered as input values for Gray level co-occurrence matrix (GLCM), where GLCM provides the spatial relationship among the pixel values of decomposed components in different directions at certain distances. The GLCMs of decomposed components are computed in (1). (0, π/2, π, 3π/2), (2). (π/4, 3π/4, 5π/4, 7π/4) (3). (0, π/4, π/2, 3π/4, π, 3π/2, 5π/4, 7π/4) directions individually and subsequently a texture feature descriptor is constructed by computing statistical parameters from the corresponding GLCMs. The retrieval performance is validated on two standard radar remote sensing image databases: 20-class satellite remote sensing dataset and 21-class land-cover dataset. The average metrices i.e., precision, recall and F-score are 61.43%, 12.29% and 20.47% for 20-class satellite remote sensing dataset while 21-class land-cover dataset have achieved 67.75%, 9.03% and 15.94% average metrices. The retrieved results show the better accuracy as compared to the other related state of arts radar remote sensing image retrieval methods.

OSAP‐Loss: Efficient optimization of average precision via involving samples after positive ones towards remote sensing image retrieval

AbstractIn existing remote sensing image retrieval (RSIR) datasets, the number of images among different classes varies dramatically, which leads to a severe class imbalance problem. Some studies propose to train the model with the ranking‐based metric (e.g., average precision [AP]), because AP is robust to class imbalance. However, current AP‐based methods overlook an important issue: only optimising samples ranking before each positive sample, which is limited by the definition of AP and is prone to local optimum. To achieve global optimisation of AP, a novel method, namely Optimising Samples after positive ones & AP loss (OSAP‐Loss) is proposed in this study. Specifically, a novel superior ranking function is designed to make the AP loss differentiable while providing a tighter upper bound. Then, a novel loss called Optimising Samples after Positive ones (OSP) loss is proposed to involve all positive and negative samples ranking after each positive one and to provide a more flexible optimisation strategy for each sample. Finally, a graphics processing unit memory‐free mechanism is developed to thoroughly address the non‐decomposability of AP optimisation. Extensive experimental results on RSIR as well as conventional image retrieval datasets show the superiority and competitive performance of OSAP‐Loss compared to the state‐of‐the‐art.

A New Remote Sensing Image Retrieval Method Based on CNN and YOLO

<>Retrieving remote sensing images plays a key role in RS fields, which activates researchers to design a highly effective extraction method of image high-level features. However, despite the advanced features of the image can be extracted by deep-learning method, features fail to cover the overall information due to its rich and complex background. Extract key regions from RS images, recede background interference and retrieval accuracy needs to be solved. Combined YOLOv5 target recognition algorithm with deep-learning method, this paper proposes a novel retrieval method based on target critical region detection. Firstly, the key retrieval regions have been identified. YOLOv5 target recognition algorithm has been used to identify the key regions of the image and served as the retrieval regions. Secondly, the retrieval characteristics are determined. Combining with the CNN model ResNet50, the retrieval features are extracted from the retrieval regions acquired in the previous step, in addition, PCA method has been used to reduce the dimension of the retrieval features. Finally, using weighted distance based on class probability to measure the similarity between a query and retrieve images. Experimental results show that the proposed method can extract better image retrieval features and improve the retrieval performance of RS image.<>

Adaptive hash code balancing for remote sensing image retrieval

ABSTRACT Remote sensing image retrieval (RSIR) refers to finding images from an image database that contain the same instance as the query image, which is an essential task in remote sensing applications. Traditional depth-based hashing algorithms usually convert the image library into a hash matrix with a specified number of digits. On the one hand, the quality of hash matrices generated by traditional methods is low and cannot guarantee good clustering between pictures of the same class. On the other hand, the ability to extract features using backbone networks must be improved. This paper proposes a deep hashing model called adaptive hash code balancing (AHCB) to solve two existing problems. The model introduces a balanced binary method to maximize the hash value entropy so that the generated hash has better clustering. Graph convolutional networks(GCNs) are introduced to automatically detect relevant data points in the graph, perform back-propagation, and propagate the updated network feedback to the feature extraction layer to improve the ability to extract features. It enables the model to learn the intrinsic data structure between remote sensing images. Experimental results on three public datasets show that the proposed method outperforms the current state-of-the-art deep hashing remote sensing image retrieval algorithms by a large margin.

Cross-domain remote sensing image retrieval with Gabor-based CNN

ABSTRACT Domain adaptation is the ability to improve the learning efficiency of the target domain by using the prior knowledge of the source domain. When applied to new tasks in the target domain, the performance of domain adaptation trained in the source domain declines sharply. The purpose is to realize a new retrieval task in the target domain using only a small number of labelled data samples, with the aid of the prior knowledge learned in the source domain. This paper focuses on semi-supervised domain adaptation in remote sensing image retrieval. The contributions of this paper are threefold. First, we construct Gabor-based CNNs to facilitate the networks to effectively capture the texture information of images. Second, we propose a cross-domain knowledge transfer strategy based on dual Gabor neural network learning. Third, we propose an unsupervised random feature mapping method based on probability distance. A large number of experiments have been conducted on UCM, WHU-RS, RSSCN7, AID, and PatternNet datasets. The results show that this method greatly improves the retrieval accuracy on the target domain and obtains state-of-the-art retrieval accuracy. The source code is available at http://nave.vr3i.com.